TMH: Two-Tower Multi-Head Attention neural network for CTR prediction.

Click-through rate (CTR) prediction is a term used to predict the probability of a user clicking on an ad or item and has become a popular research area in advertising. As the volume of Internet data increases, the labor costs of traditional feature engineering continue to rise. To reduce the dependence on feature interactions, this paper proposes a fusion model that combines explicit and implicit feature interactions, called the Two-Tower Multi-Head Attention Neural Network (TMH) approach. The model integrates multiple components such as multi-head attention, residual network, and deep neural networks into an end-to-end model that automatically obtains vector-level combinations of explicit and implicit features to predict click-through rates through higher-order explicit and implicit interactions. We evaluated the effectiveness of TMH in CTR prediction through numerous experiments using three real datasets. The results demonstrate that our proposed method not only outperforms existing prediction methods but also offers good interpretability.

A ResNet-LSTM hybrid model for predicting epileptic seizures using a pretrained model with supervised contrastive learning.

In this paper, we propose a method for predicting epileptic seizures using a pre-trained model utilizing supervised contrastive learning and a hybrid model combining residual networks (ResNet) and long short-term memory (LSTM). The proposed training approach encompasses three key phases: pre-processing, pre-training as a pretext task, and training as a downstream task. In the pre-processing phase, the data is transformed into a spectrogram image using short time Fourier transform (STFT), which extracts both time and frequency information. This step compensates for the inherent complexity and irregularity of electroencephalography (EEG) data, which often hampers effective data analysis. During the pre-training phase, augmented data is generated from the original dataset using techniques such as band-stop filtering and temporal cutout. Subsequently, a ResNet model is pre-trained alongside a supervised contrastive loss model, learning the representation of the spectrogram image. In the training phase, a hybrid model is constructed by combining ResNet, initialized with weight values from the pre-trained model, and LSTM. This hybrid model extracts image features and time information to enhance prediction accuracy. The proposed method's effectiveness is validated using datasets from CHB-MIT and Seoul National University Hospital (SNUH). The method's generalization ability is confirmed through Leave-one-out cross-validation. From the experimental results measuring accuracy, sensitivity, and false positive rate (FPR), CHB-MIT was 91.90%, 89.64%, 0.058 and SNUH was 83.37%, 79.89%, and 0.131. The experimental results demonstrate that the proposed method outperforms the conventional methods.

DRL-OS: A Deep Reinforcement Learning-Based Offloading Scheduler in Mobile Edge Computing.

Hardware bottlenecks can throttle smart device (SD) performance when executing computation-intensive and delay-sensitive applications. Hence, task offloading can be used to transfer computation-intensive tasks to an external server or processor in Mobile Edge Computing. However, in this approach, the offloaded task can be useless when a process is significantly delayed or a deadline has expired. Due to the uncertain task processing via offloading, it is challenging for each SD to determine its offloading decision (whether to local or remote and drop). This study proposes a deep-reinforcement-learning-based offloading scheduler (DRL-OS) that considers the energy balance in selecting the method for performing a task, such as local computing, offloading, or dropping. The proposed DRL-OS is based on the double dueling deep Q-network (D3QN) and selects an appropriate action by learning the task size, deadline, queue, and residual battery charge. The average battery level, drop rate, and average latency of the DRL-OS were measured in simulations to analyze the scheduler performance. The DRL-OS exhibits a lower average battery level (up to 54%) and lower drop rate (up to 42.5%) than existing schemes. The scheduler also achieves a lower average latency of 0.01 to >0.25 s, despite subtle case-wise differences in the average latency.

An XAI method for convolutional neural networks in self-driving cars.

eXplainable Artificial Intelligence (XAI) is a new trend of machine learning. Machine learning models are used to predict or decide something, and they derive output based on a large volume of data set. Here, the problem is that it is hard to know why such prediction was derived, especially when using deep learning models. It makes the models unreliable in the case of reliability-critical applications. So, it is required to explain how they derived such output. It is a reliability-critical application for self-driving cars because the mistakes made by the computers inside them can lead to critical accidents. So, it is necessary to adopt XAI models in this field. In this paper, we propose an XAI method based on computing and explaining the difference of the output values of the neurons in the last hidden layer of convolutional neural networks. First, we input the original image and some modified images of it. Then we derive output values for each image and compare these values. Then, we introduce the Sensitivity Analysis technique to explain which parts of the original image are needed to distinguish the category. In detail, we divide the image into several parts and fill these parts with shades. First, we compute the influence value on the vector indicating the last hidden layer of the model for each of these parts. Then we draw shades whose darkness is in proportion to the influence values. The experimental results show that our approach for XAI in self-driving cars finds the parts needed to distinguish the category of these images accurately.

A LSTM-Hawkes hybrid model for posterior click distribution forecast in the advertising network environment.

In the field of advertising technology, it is a key task to forecast posterior click distribution since 66% of advertising transactions depend on cost per click model. However, due to the General Data Protection Regulation, machine learning techniques to forecast posterior click distribution based on the sequences of an identified user's actions are restricted in European countries. To overcome this barrier, we introduce a contextual behavior concept for the advertising network environment and propose a new hybrid model, which we call the Long Short Term Memory-Hawkes model by combining a stochastic-based generative model and a machine learning-based predictive model. Also, to meet the computational efficiency for the heavy demand in mobile advertisement market, we define gradient exponential kernel with just three hyper parameters to minimize residuals. We have carefully tested our proposed model with production data and found that the LSTM-Hawkes model reduces the Mean Squared Error by at least 27.1% and up to 83.8% on average in comparison to the existing Hawkes Process based algorithm, Hawkes Intensity Process, as well as 39.77% on average in comparison to Multivariate Linear Regression. We have also found that our proposed model improves the forecast accuracy by about 21.2% on average.

F-DCS: FMI-Based Distributed CPS Simulation Framework with a Redundancy Reduction Algorithm.

A cyber physical system (CPS) is a distributed control system in which the cyber part and physical part are tightly interconnected. A representative CPS is an electric vehicle (EV) composed of a complex system and information and communication technology (ICT), preliminary verified through simulations for performance prediction and a quantitative analysis is essential because an EV comprises a complex CPS. This paper proposes an FMI-based distributed CPS simulation framework (F-DCS) adopting a redundancy reduction algorithm (RRA) for the validation of EV simulation. Furthermore, the proposed algorithm was enhanced to ensure an efficient simulation time and accuracy by predicting and reducing repetition patterns involved during the simulation progress through advances in the distributed CPS simulation. The proposed RRA improves the simulation speed and efficiency by avoiding the repeated portions of a given driving cycle while still maintaining accuracy. To evaluate the performance of the proposed F-DCS, an EV model was simulated by adopting the RRA. The results confirm that the F-DCS with RRA efficiently reduced the simulation time (over 30%) while maintaining a conventional accuracy. Furthermore, the proposed F-DCS was applied to the RRA, which provided results reflecting real-time sensor information.

I-DWRL: Improved Dual Wireless Radio Localization Using Magnetometer.

In the dual wireless radio localization (DWRL) technique each sensor node is equipped with two ultra-wide band (UWB) radios; the distance between the two radios is a few tens of centimeters. For localization, the DWRL technique must use at least two pre-localized nodes to fully localize an unlocalized node. Moreover, in the DWRL technique it is also not possible for two sensor nodes to properly communicate location information unless each of the four UWB radios of two communicating sensor nodes cannot approach the remaining three radios. In this paper, we propose an improved DWRL (I-DWRL) algorithm along with mounting a magnetometer sensor on one of the UWB radios of all sensor nodes. This addition of a magnetometer helps to improve DWRL algorithm such that only one localized sensor node is required for the localization of an unlocalized sensor node, and localization can also be achieved even when some of the four radios of two nodes are unable to communicate with the remaining three radios. The results show that with the use of a magnetometer a greater number of nodes can be localized with a smaller transmission range, less energy and a shorter period of time. In comparison with the conventional DWRL algorithm, our I-DWRL not only maintains the localization error but also requires around half of semi-localizations, 60% of the time, 70% of the energy and a shorter communication range to fully localize an entire network. Moreover, I-DWRL can even localize more nodes while transmission range is not sufficient for DWRL algorithm.

International live endoscopic multichannel demonstration using superfast broadband internet connections.

BACKGROUND/AIMS: Telemedicine is a convenient and efficient tool for remote education in various fields. The telemedicine system can also be used to educate doctors and medical students. The aim of our study was to establish the effectiveness of the telemedical system for use in a live endoscopic multichannel demonstration conference and to test the effectiveness and usefulness of a multicenter-based live endoscopic demonstration through live, interactive, high resolution video transmission using advanced networks and the digital video transport system (DVTS). METHODS: This study is a prospective multicenter pilot study. A live demonstration of an endoscopic submucosal dissection (ESD) and an endoscopic retrograde cholangiopancreatography (ERCP) using advanced network technology was performed. RESULTS: The DVTS successfully transmitted uncompressed, high-resolution, digital lectures with endoscopy video during a multichannel endoscopic live demonstration of ESD and ERCP over multiple advanced networks. The overall satisfaction rating when the endoscopic lecture demonstration was performed by combining DVTS was generally good. CONCLUSIONS: We believe that a multicenter-based live endoscopic demonstration is a very effective conferencing method when using advanced networks and DVTS.